A Note On the Connection and Equivalence of Three Sparse Linear Discriminant Analysis Methods

In this paper we reveal the connection and equivalence of three sparse linear discriminant analysis methods: the `1-Fisher’s discriminant analysis proposed in Wu et al. (2008), the sparse optimal scoring proposed in Clemmensen et al. (2011) and the direct sparse discriminant analysis proposed in Mai et al. (2012). It is shown that, for any sequence of penalization parameters, the normalized solutions of direct sparse discriminant analysis equal the normalized solutions of the other two methods at different penalization parameters. A prostate cancer dataset is used to demonstrate the theory.